Strengths and Weaknesses of Parallel Tempering 

J. Machtjf] 

Physics Department, University of Massachusetts, Amherst, MA 010003 USA 

Abstract 

Parallel tempering, also known as replica exchange Monte Carlo, is studied in the context of 
two simple free energy landscapes. The first is a double well potential defined by two macrostates 
separated by a barrier. The second is a 'golf course' potential defined by microstates having two 
possible energies with exponentially more high energy states than low energy states. The equili- 
bration time for replica exchange is analyzed for both systems. For the double well system, parallel 
tempering with a number of replicas that scales as the square root of the barrier height yields expo- 
nential speedup of the equilibration time. On the other hand, replica exchange yields only marginal 
speed-up for the golf course system. For the double well system, the free energy difference between 
the two wells has a large effect on the equilibration time. Nearly degenerate wells equilibrate much 
more slowly than strongly asymmetric wells. It is proposed that this difference in equilibration time 
may lead to a bias in measuring overlaps in spin glasses. These examples illustrate the strengths 
and weaknesses of replica exchange and may serve as a guide for understanding and improving the 
method in various applications. 
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I. INTRODUCTION 



Replica exchange Monte Carlo (MC), also known as parallel tempering, was independently 
introduced by Swendsen and Wang [IJ and Hukushima and Nemoto [2] for studying spin 
glasses. Replica exchange MC is an important tool in many areas of computational physics [3] 
where the free energy landscape has many metastable minima separated by barriers. It is 
the standard method for simulating spin glasses and is used for protein folding [U |5] and 
lattice gauge theory [6]. 

In parallel tempering many replicas of the system are simulated in parallel using a stan- 
dard MC technique for sampling the Gibbs distribution such as the Metropolis-Hastings 
algorithm. The replicas are simulated at different temperatures. The fixed sequence of tem- 
peratures extends from some low temperature where the equilibration time is very long to 
some high temperature where the equilibration time is short. Replica exchange moves per- 
mit replicas at adjacent temperatures to swap temperatures in a way that satisfies detailed 
balance so that the entire set of replicas equilibrates at the prescribed set of temperatures. 
The heuristic motivation for replica exchange is that replicas can diffuse from the lowest 
to the highest temperature and back to the lowest temperature. During this 'roundtrip' 
equilibration occurs at high temperature so that when the replica returns to the lowest tem- 
perature its state is independent of the original state. A number of studies have focused 
on optimizing replica exchange MC by choosing the set of replica temperatures and other 
parameters to minimize the round-trip time [3 El E] ■ Replica exchange MC is also closely 
related both to simulated annealing and various generalized ensemble methods [10] . 

In the present paper we consider the efficiency of replica exchange MC in the context 
of two very simple free energy landscapes that, respectively, highlight the strengths and 
weaknesses of the method. The first example is a free energy landscape with two minima 
separated by a barrier such as occurs in the 0^ free energy functional. In the low temperature 
phase where the potential has a double well, local dynamics fully equilibrates in a time that 
is exponential in the barrier height though equilibration within a single well is typically 
much faster. As we shall see, replica exchange can reduce the barrier crossing time from 
exponential to a polynomial in the barrier height. 

For the double well potential, we show that the equilibration time is longest when the 
free energy difference between the wells is a few ksT or less. In this situation, the Gibbs 
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measure gives significant weight to both macrostates and equihbration involves diffusive 
motion of rephcas. When the free energy difference between the wells is much larger than 
ksT, equilibration depends on much faster ballistic motion of replicas. In the context of 
spin glasses, a free energy landscapes with multiple free energy minima is expected. In the 
discussion section, we argue that the longer equilibration time for nearly degenerate free 
energy minima may lead to an overestimate of the probability that the spin overlap in spin 
glasses is near zero. This reasoning suggests caution in interpreting simulations for the spin 
overlap. 

The second free energy landscape to be considered is the 'golf course' potential. Here 
almost all of the microstates are degenerate excited states and an exponentially small frac- 
tion of states are degenerate ground states. This model has a pseudo-first-order transition 
between a low temperature phase where the system is almost always in a ground state and 
a high temperature phase where the system is almost always in an excited state. For this 
particularly nasty system, replica exchange is of little help equilibrating the low temperature 
phase. The equilibration time is controlled by the time taken to find a ground state and this 
time is not reduced by bringing the system first to a higher temperature. 

The outline of the paper is as follows. In Sec. [IT] we introduce replica exchange Monte 



Carlo Section. In Sec. Ill we analyze the behavior of replica exchange Monte Carlo for the 
double well free energy landscape both analytically and via numerical simulations. In Sec. 



IV we introduce the golf course potential and analyze replica exchange for this potential. 



The paper closes with a discussion in Sec. IVl 



II. REPLICA EXCHANGE MONTE CARLO 



Replica exchange Monte Carlo is designed to equilibrate a set of replicas of a system at 
inverse temperatures, 

/?o>A>...,/3ii-i (1) 

Each of the R replicas is equipped with dynamics that equilibrates it at its respective tem- 
perature. In addition, replica exchange moves are permitted between replicas at neighboring 
temperatures. In a replica exchange move, the temperatures of the two replicas are swapped. 
In order to satisfy detailed balance and insure that the entire set of replicas equilibrates, the 
probability for accepting a replica exchange move between replicas at temperatures P and 
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(2) 



(5' is given by 

where E and E' are the respective energies of the rephcas that were originally at (3 and /?'. 
If the replica exchange move is accepted, the replica whose local dynamics was set at inverse 
temperature (5 is now set at (5' and vice versa. 



III. DOUBLE WELL POTENTIAL 



In this section we consider a simple free energy with two minima such as occurs, for 
example, in a 0^ theory. The free energy associated with each minima or well is, 

(3FM--\{P-^f{K + Ha) (3) 

where a labels the well; the deep well is indicated by u = 1 and the shallow well by cr = 0. 
Although the free energy landscape itself is not explicitly prescribed, we assume that free 
energy at the saddle point between the wells is zero so that F is the free energy barrier for 

transitions between wells. From the free energy we can obtain the internal energy U„{f3) 
and energy fluctuations by differentiations with respect to /5. The internal energy, which is 
the expectation of the energy E is 

U^iP) = E{E) = -{13 - P,){K + Ha) (4) 

and the variance of the energy is 

= Yav{E) ^{K + Ha) (5) 

The free energy difference between the wells is controlled by if (if > 0) and given 

by, 

(35F(P) = (3Fo{(3) - (3F,((3) = ^((3 - (3,fH (6) 

Given this free energy difference, the probability c{j3) of being in the deep well (i. e. the 
expectation of a) at inverse temperature {3 is 

To completely specify the statics of the model, we assume that the energy distribution in 
each well is a normal distribution with mean Ufj{(3) and variance A^. 
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A. Replica Exchange Dynamics for the Double Well Model 

We suppose that each rephca is equipped with single temperature dynamics that is much 
faster than the rate of rephca exchange attempts. Thus, for each rephca exchange attempt, 
the energies of the two rephcas are chosen independently from normal distributions for the 
given replica's temperature and well index. The time scale for transitions between wells by 
single temperature dynamics for j3 < jSc is order e~^^ , however, in the analysis and simula- 
tions that follow, we do not permit single temperature dynamics to effect transitions between 
the wells except at jSc- This simplification leads to an underestimate of the equilibration 
time for replica exchange since the replicas near (3^ may also contribute to barrier crossings 
between the wells. Since the entropy of each well is the same dX j3 = (3c and since there is 
no barrier between the wells here, we assume an initial condition for parallel tempering in 
which each well is equally hkely to be populated for every (3 < Pe- 
lt is straightforward to verify that the replica exchange dynamics described above satisfies 
detailed balance with respect to the normal distribution of the energies in the two wells and 
the probability c{(3) given in for being in the deep well. The normal distribution of 
energies within each well is maintained by fiat while c(/3) is established via replica exchange. 
Our goal is to understand the time scale for reaching this equilibrium well distribution. 

Given the dynamical assumptions it is not difficult to calculate the average rate of replica 
exchange for the double well model. Let Wo-,o-'(/5, P') be the average rate of replica exchange 
if the two replicas are, respectively, at inverse temperatures (3 and (3' in wells a and a'. 
Without loss of generality, assume /3 > /?'. The rate yVa,a'{P, P') is obtained by averaging 
(|2| over the energy distribution, 



Here E(-) is an average over the normal distributions of E and E', the energies in the 
respective wells at the given temperatures. The explicit expression for the replica exchange 
rate is 




(8) 



yv.y{P,p') = 




g-(i?-C/,(/3))2/2A2-(£'-C/,,(/3'))V2A2, 



(9) 
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B. Degenerate wells: H = 



First consider the simpler case of degenerate wells {H = 0), it is not hard to show that 
the two terms in ^ are equal and, using Q and (|5]), the Gaussian integrals yield, 



W.AP, P') = Erfc l^iL^h^j (10) 

where Erfc(-) is the complementary error function. The replica exchange rates are indepen- 
dent of the well indices and the dynamics of replica exchange is diffusive. 

Given the replica exchange rate, we can estimate the equilibration time as follows. Sup- 
pose there are R equally spaced replicas with inverse temperatures ranging from j3o to (3c- 
Equilibration requires that a replica in one well at the lowest temperature f3o diffuses to f3c 
where the well is randomized. Up to constant factors, the equilibration time t{R) for R 
replicas scales like the mean first passage time for a random walk between the ends of a 
chain of R sites with hopping rate W, with a reflecting boundary at Po and an absorbing 
boundary at Pc- The mean first passage time for this process is {R — 1)^/W [13j. Thus, 



from (10) 



r(fl)~(fl-l)7ErJ 'y:y ) (11) 

From the asymptotic behavior of the error function, Erfc(x) ~ exp(— we see 
that the optimum number of replicas should scale as the square root of the well depth, 
-Ropt ~ i/3o — (3c)^/K. The rephca exchange rate is then order unity and the optimized 
equilibration time in this diffusive regime, is proportional to the well depth, 

r^~(i?opt-l)'~m-/5c)'. (12) 



Since the optimum number of replicas is independent of prefactors in (11), we can obtain 



i?opt by numerically minimizing the RHS of (11) with respect to R, the result is 



i?opt = 1 + 0.594(;3o - PcWK. (13) 

The above result represents the main strength of replica exchange Monte Carlo. The 
time for barrier crossing between the wells has been reduced from an exponential in the 
barrier height to linear in the barrier height. Note that a key feature of the double well 
model required for the success of parallel tempering is the continuity of the free energy with 
temperature. 



C. Asymmetric Wells: H > 



When H > 0, the wells are asymmetric and the motion of replicas is biased diffusion. 
Replicas in the deep well move toward lower temperatures relative to replicas in the shallow 
well. The average replica exchange rates reflect that bias and, carrying out the integrals in 
(|9|, we obtain 

Wo,i(/3,/3') =^1+^2^3 (14) 



and 



where 



Wi,o(/?,/?')=^2+^l/^3 (15) 



and 



2 V V4i^ + 2H y ' ^ ^ 

£s = exp (^iP - P')i^^ - ■ (18) 

Since £i > £2 and £^3 > 1, we have Wo,i(/?,/3') > Wi,o(/9,/9') and the dynamics is biased 
toward deep well replicas moving to lower temperatures. The velocity V(/3,/3') of deep well 
(shallow well) replicas toward lower (higher) temperature is the difference of the rates, 

V = >Vo,i - >Vi,o (19) 

Figure [l] shows V vs A/? = P — P' for several values of H and the choice K = 16, P = 5 
and Pc = 1- The three curves correspond to if = 2 (bottom), H = 5 (middle) and H = 20 
(top). The qualitative features are that velocity increases as the bias, H increases and that 
Ap must neither be too large and nor too near zero to maximize the velocity. As the bias 
increases, the velocity approaches unity for an increasing range of A/3. 

As H increases there is a complicated crossover from diffusion to biased diffusion to bal- 
listic motion. Unlike the symmetric case, for H > the arguments of the error functions 
depend on both the temperature difference between replicas and the absolute temperature so 
that evenly spaced replicas cannot be expected to optimize the equilibration time. Nonethe- 
less, we can make some crude estimates for the ballistic regime. The requirement for the 
ballistic regime is that P6F = {P — PcYH/2 ^ 1 for most replica temperatures P so that 
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FIG. 1: The velocity V vs. the temperature between the rephcas A/3 = /? — /?' for three values of 
the well asymmetry parameter, H = 2 (blue, bottom), 5 (green, middle) and 20 (red, top) for the 
choice K = 16, /? = 5 and /3c = 1. 

each replica is nearly always in the deep well. Then A/5 can be chosen so that V ~ 1. In this 
regime the equilibration time scale is simply the time required to generate order R states in 
the deep well at (3^ and then move them to the colder replicas. This time scale is order R. 
The argument of the error function must be small to achieve a velocity near unity, which is 
essentially the same condition as in the diffusive regime to that -Ropt ~ (/?o — (3c)y/K. The 
behavior of the equilibration time in the ballistic regime is expected to be 



The main point here is that the time scale is much longer for the diffusive regime than the 
ballistic regime. In the diffusive regime, the equilibration time is proportional to the free 
energy barrier between the wells but in the ballistic regime it behaves as the square root of 
the free energy barrier. 

We have so far considered the simple situation where the free energy difference between 
the wells changes monotonically with the temperature-one well is the deeper than the other 
for all (3 > (3c- If instead, the free energy difference between wells changes sign in a tem- 
perature region where there is a large barrier between the wells then the motion of replicas 




(20) 
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will be biased in a way that causes trapping and very long equilibration times. This is the 
situation that holds at a thermal first-order transition. It would be interesting to calculate 
the equilibration times in a simple free energy landscape with a first-order transition. 



D. Numerical Simulations 

We carried out simulations of parallel tempering for the double well model. In each 
step of the simulation, a pair of adjacent temperatures (3 and (3' is randomly chosen. The 
energies of the associated replicas are chosen from normal distributions with means Ua{j3) 
and Ua'iP') and standard deviations A^- and A^', respectively. The replica exchange move 
is accepted with probability given by ([2]). If one of the replicas is at Pc, the well indicator a 
for this replica is chosen randomly before attempting the replica exchange move. Otherwise, 
transitions between the two wells are forbidden and a is conserved. One MC sweep consists 
of i? — 1 replica exchange attempts and time is measured in sweeps. The simulations are 
initialized so that each replica is randomly chosen to be in either well with equal probability. 

We simulated several values of the parameters H, K, and R. In all simulations we chose 
Po = 5 and /5c = 1 so as to be deep in the low temperature regime. We measured two 
quantities, the exponential autocorrelation time and the initial decay toward equilibrium. 
The exponential autocorrelation time Texp is obtained from the autocorrelation function, 
r(t) of the fraction of rephcas in the deep well p = (1/-R) X]£o^ 

Here (■) indicates an equilibrium average and the measurement of p{t) is displaced by t sweeps 
from p. The equilibrium average is obtained from long time averages. The initialization 
time before data collection is typically several thousand sweeps and the run time is 10 to 
50 million sweeps. Except for an initial period of faster decay, T{t) is well described by a 
single exponential and the exponential autocorrelation time Xexp is obtained by fitting to a 
single exponential function, T{t) = ae"*/'^°''p over an appropriate range of t. We considered 
= 8, 16, 32 and 64 and H = 0, 0.5, 1, 2, 3, 5 and 10. We explored a range of numbers 



of replicas R to find -Ropt- We found that -Ropt is correctly predicted by (13) for if = and 
that for if > 0, -Ropt is slightly larger than for the symmetric H = case but within one 
or two of the predictions of (13). An exact measurement of -Ropt for the asymmetric case 
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FIG. 2: The exponential autocorrelation time Texp for the fraction of sites in the deep well vs. the 
well depth parameter K for = 10 (green diamonds), H = 5 (red squares) and H = (blue 
circles). The lines are best power law fits, Texp ~ with x = 0.76, 0.83 and 0.99 for H = 10, 5 
and 0, respectively. 

proved difficult because t{R) varies very little with R near -Ropt- 

Figure [2] shows Texp vs. i^' for iJ = 0, 5 and 10 and for K = 8, 16, 32 and 64. Statistical 
errors are smaller than the data points. The lines are best power law fits of the form aK^. 
The fitted powers are x = 0.76, 0.83 and 0.99 for H = 10, 5 and 0, respectively. The fact 



that Texp increases essentially linearly in K for H = agrees with (12) of Sec. IIIB for 



symmetric wells. However, although x < 1 for the two asymmetric cases, we do not observe 
X = 0.5, as predicted in (20) of Sec. IIIC for highly asymmetric wells, even for H = 10. We 



believe this is a crossover effect but it may also indicate more subtleties in the asymmetric 
case than have been taken into account in the simple theoretical arguments based on ballistic 
motion of replicas. Figure [3] shows Toxp vs H for fixed i^' = 16 revealing the rapid decline in 
equilibration time as the asymmetry increases. 

The initial relaxation to equilibrium of the lowest temperature replica is often more 
relevant for applications of parallel tempering than the exponential autocorrelation time. 
To study the initial relaxation to equilibrium, we investigated 7(t), the probability that the 
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FIG. 3: The exponential autocorrelation time Texp vs. the asymmetry between the wells H for 
K = 16. 



lowest temperature replica is in the deep well after t sweeps, 



7(t) = (ao(t))-c(/3o). 



(22) 



Note that (cro(O)) = 1/2 for the standard initial condition and limt^oo(o"o(^)) = c(/3o) so 
that 7(t) approaches zero for large t. The error in sampling (Tq after t sweeps is determined 
by 7(t). Figures |4] shows loglinear plots of 7(t) as a function of t for (a) small asymmetry 
H = 0.1 and (b) large asymmetry H = 5 for K = 16 and R = 12. For small asymmetry, 7(t) 
decays nearly exponentially after an initial faster decay with a time scale that is very close 
to Texp. For large asymmetry, two time scales are clearly apparent. The short time scale is 
approximately 2 and the long time scale is about 9, whereas Texp = 11 for these parameters. 
Presumably, the time scale r^^p would finally be apparent in 7(t) but perhaps not until it 
has decayed to extremely small values. 

The initial short time scale for 7(t) for the strongly asymmetric H = 5 case can be 
understood qualitatively as the average time for the lowest temperature replica that is 
also in the deep well to move to the lowest temperature /3o- If this nearest replica is at 
temperature jSk the expected time for it to move to (3q is approximately k/V where V is the 



velocity defined in ( 19 ), which is nearly unity in the strongly asymmetric case for sufficiently 
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closely spaced replicas. Thus we expect that initially, 7(t) will decay on a time scale order 
unity. Since this initial decay persists to quite small values of •yit) it may be the relevant 
time scale for practical equilibration in the strongly asymmetric case. 

The key finding of the numerical study is that equilibration times are much shorter for 
asymmetric free energy minima than for nearly degenerate minima. 

IV. GOLF COURSE LANDSCAPE 

In this section we consider the efficacy of replica exchange for the 'golf course' landscape. 
Like a flat golf course, this landscape has a small number of degenerate ground states-the 
'hole '-and an exponentially larger number of degenerate excited states-the 'green.' The 
golf-course landscape has microstates where N is the 'system size.' A fraction e"^"'^^ 
of these microstates have energy while the remaining states have energy eA^ with e > 0. 
The golf course system is a quenched disordered system; different realizations of disorder 
correspond to different sets of ground states. The microstates of the system are labeled by 
integers and we suppose that there is an oracle for each realization of the system that tells 
whether a given integer label corresponds to a ground state. On the other hand, the set of 
ground states is itself inaccessible except by exhaustive search. Natural realizations of golf 
course landscapes are studied in [2]. 

The parameter (3c is also the inverse temperature of a pseudo-first-order transition. For 
l3 > Pc and large N the system is almost surely in a ground state but for /? < /?c it is almost 
surely in an excited state. More specifically, let c(/3) be the probability of being in a ground 
state at temperature /?. It is straightforward to see that c(/3) is given by 

^(^) = i + e-(V/3.).^ - 

The dynamics of the system is a global version of the Metropolis-Hastings algorithm. 
Each step consists of proposing a random microstate, consulting the oracle to determine its 
energy and then accepting the proposal with probability, min [l, e-i^^^] where AE is the 
difference in energy between the proposed and initial microstate. This dynamics converges 
to equilibrium, however, the equilibration time is exponential in N. In units of MC steps, 
the excitation rate from a ground state to an excited state is controlled by the energy 
barrier e~^^^. The de-excitation rate from an excited state to a ground state is controlled 
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FIG. 4: The initial approach to equiHbrium of the probabiHty of the coldest replica being in the 
deep well j{t) vs time t after the initial conditions for (a) small asymmetry H = 0.1 and (b) large 
asymmetry H = 5.0. In both cases K = IQ, Pq = ^ and R = 12. 
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by the entropy barrier is e'^""^^ . The equihbration rate is the sum of these two rates, 
^-PceN _|_ g-/3eAf^ Yoi (3q > (3c the equihbration rate is dominated by the de-excitation term 
and the equihbration time Tq is approximately, 

To ^ e-^^^^ (24) 

On the other hand, for /3i < (3^-, the equihbration rate is dominated by the excitation rate 
and the equihbration time ti is 

n ^ e"^^^^ (25) 

Suppose we wish to equihbrate a system at inverse temperature /3o > Pc using rephca 
exchange Monte Carlo with replicas given by ([T| with (3r^i < Pc- Initially each replica is 
almost certainly in an excited state. Since all replicas have the same energy, replica exchange 
moves are always accepted and the round trip time is independent of N though proportional 
to R^. Of course, this short initial round trip time is misleading and has nothing to do with 
the equilibration time. Equilibration requires finding ground states and this happens on the 
exponential time scale tq. If a ground state is discovered by a high temperature replica, 
that ground state will quickly and nearly irreversibly be passed to lower temperature by 
replica exchange. If there are £ replica temperatures in the low temperature 'phase,' that 
is, if Pi^i < (3c and (3i > (3c then a ground state must be discovered £ times to populate 
each cold replica. Since there are R systems looking for the ground state, we obtain a 
modest acceleration of order R/£. For example, if £ = 1, there is a factor of R speed-up 
due to replica exchange. This speed-up is not dependent on faster equilibration at high 
temperatures but relies on simple parallelism; all replicas are put to work independently 
looking for rare ground states but only I ground states need to be found. In conclusion, for 
the golf course landscape, rephca exchange achieves a modest speed-up in the equilibration 
time due to brute force parallelism. Before equilibration has been achieved, the round-trip 
time is short and unrelated to the equilibration time. The golf course landscape is the most 
extreme case of the problem of the equilibrium macrostate having an exponentially small 
and hidden basin of attraction. 
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V. DISCUSSION 



The broad conclusion of this work is that rephca exchange Monte Carlo is efficient for 
systems with free energy landscapes with multiple minima so long as the landscape varies 
continuously and monotonically with temperature and the relevant minima have large basins 
of attraction. In this situation, replica exchange is able to quickly sample the minima with 
the correct weighting. In this context, a basin of attraction of a macrostate is imprecisely 
defined as the subset of microstates that has a high likelihood of reaching the macrostate 
via a quench from high temperature. The speed of the quench must be much slower than 
the rate of equilibration within the macrostate and much faster than the transition rate 
between macrostates. Given the assumption of large basins of attraction and exponential 
barriers between macrostates, parallel tempering with polynomially many replicas reaches 
equilibrium in a time that is polynomial in the barrier height and thus achieves exponential 
speed-up. On the other hand, replica exchange yields little improvement for systems where 
the relevant macrostates states have small basins of attraction. Here the problem is simply 
finding equilibrium states rather than moving between them. For real world applications, 
both kinds of problems may be present-barriers between multiple states and small basins of 
attraction. 

Let's now consider the case of Ising spin glasses in three dimensions. It is is known that 
finding ground states is NP-hard [T3]. This fact suggest, but does not prove, that the basins 
of attraction of the low temperature equilibrium states are exponentially small. On the other 
hand, for temperatures not too far below the critical temperature it may be that the basins 
of attraction are still relatively large and replica exchange can produce large reductions in 
equilibration times. 

As a working hypothesis, let us adopt the droplet picture [HI [12] for the low temperature 
phase of the three-dimensional Ising spin glass. Within the droplet picture something like 
the double well model should describe the lowest lying states in the low temperature phase. 
The two wells correspond to the two orientations of the droplet and fluctuations around 
these orientations [27]. Each realization of disorder has different values of the barrier height 
and free energy difference. The statistics of these parameters are assumed to have power law 
behavior in the linear systems size L. Specifically, K and H'^ ~ L^^ where the overbar 
refers to a disorder average and 6 is believed to be near 0.2 for the three-dimensional Ising 
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spin glass. 

Parallel tempering has proved to be a successful tool for studying 3D spin glasses and it 
is reasonable to assume that for small systems the equilibrium states have sufficiently large 
basins of attraction that they can be "found" in a reasonable time by replica exchange. 
Even so, there is a potential source of bias in parallel tempering as it is typically used. In 
typical applications the length of the run is fixed independent of the realization of disorder. 
These parameters are chosen to insure that some disorder averages are near their equilibrium 
values. For example, the test described in insures that the disorder averaged energy is 
near its equilibrium value. However, this requirement may not guarantee that all relevant 
observables are well equilibrated. 

In the droplet model, the fraction of realizations with nearly degenerate lowest lying 
states scales as L~^. Thus most disorder realizations have a large free energy difference 
between the equilibrium state and the excited (droplet) state. In the context of the dou- 
ble well model, these realizations have (/3 — (3c)'^H/2 ^ 1 and, as we have seen, they will 
be rapidly equilibrated by parallel tempering. On the other hand, the small fraction (or- 
der L~^) of realizations with 'active droplets,' that is, two nearly degenerate minima, will 
have much longer than typical equilibration times. Both because these realizations are rare 
and because the two droplet orientations have similar energies, the poor equilibration of 
these active droplet realizations will not introduce much error in the measurement of the 
disorder averaged energy. The same cannot be said for the disorder averaged spin overlap 
distribution near zero overlap, P{q ~ 0). It is precisely the difficult to equilibrate, active 
droplet realizations that contribute to this quantity since these are the systems that have 
a significant likelihood in equilibrium of having either droplet orientation. If these realiza- 
tions are not equilibrated it will lead to an overestimate of P{q ~ 0). In particular, the 
equilibrium contribution of a given realization to P{q ~ 0) depends on the relative weight of 
the two droplet states. Given the simplifying assumption that the two droplet states have 
zero overlap, a realization with free energy difference I35F{(3) between the droplet states 
will contribute 2c(/3)(l - c(/3)) = 26-1^^^ /{I + e'^^^f to P(g ^ 0). Exactly degenerate 
disorder realizations contribute 1/2 to P{q 0) but as (35F becomes larger than unity, the 
equilibrium contribution to P{q ^ 0) diminishes rapidly. However, for times less than the 
equilibration time the two droplet orientations will be close to equally populated assuming 
both have nearly equal basins of attraction as expected in the droplet model. The result of 
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this bias is that, P{q ~ 0) approaches its equihbrium value from above on a time scale asso- 
ciated with the equilibration of realizations with active droplets. This time scale is expected 
to be considerably longer than the time scale for the equilibration of the disorder averaged 
energy. 

The nature of the low temperature phase of finite-dimensional spin glasses is the subject of 
a long standing controversy. In the droplet scenario most realizations of disorder have a large 
gap between a unique equilibrium state and a macroscopically different low lying excited 
state. The replica symmetry breaking picture [T71 UHl [121 1201 EI] proposes multiple nearly 
degenerate low lying equilibrium states with an ultrametric overlap structure. A scenario 
that has features of both pictures and is supported by numerical studies is the 'trivial non- 
trivial' (TNT) picture [221 123] . The possible state structures in the thermodynamic limit 
are constrained by mathematical theorems p3j but the various scenarios are difficult to 
distinguish in simulations of small systems. The disorder averaged spin overlap near zero, 
P{q ~ 0) has been studied numerically to distinguish these scenarios [221 1231 12S]- In the 
RSB and also the TNT an pictures, this quantity approaches a constant while in the droplet 
picture it decreases as . In simulations P{q ~ 0) decreases for small L but then reaches 
a plateau at a small but nonzero value [26]. The above considerations suggest caution 
in interpreting numerical results for P(g ^0). It would be useful to carefully study the 
correlation between the equilibration time of disorder realizations and their contribution to 
P{q ~ 0) to insure that this quantity has been correctly measured in simulations. 
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